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Overview 



Improving the success of academically underprepared students who are in need of developmental (or 
remedial) education is a key challenge facing community colleges today. Many of these students 
enter college with little awareness of these institutions’ expectations or a clear model for how to 
make effective decisions about their academic careers. To help students address these challenges, a 
number of colleges across the country have looked to success courses (also called study skills, 
student development, or new student orientation courses). This report analyzes a success course for 
developmental education students at Guilford Technical Community College in Greensboro, North 
Carolina, and its impact on students’ psychosocial s ki lls and behaviors and academic achievement. 

After joining Achieving the Dream: Community Colleges Count in 2004, a national organization 
designed to mentor colleges through an institutionwide, student success-oriented improvement 
process, Guilford chose to offer a revised version of its student success course to developmental 
education students, aimed at improving psychosocial awareness and academic achievement. 
Modeled on Skip Downing’s On Course philosophy and curriculum, it placed an intensive focus on 
changing students’ behaviors and attitudes, including increasing their awareness of their and others’ 
emotions, understanding their own learning styles, improving time management skills, and recogniz- 
ing their responsibility for their own learning. Guilford hoped that these changes in students’ 
personal habits and behaviors might help them take better control of their academic lives, which 
would ultimately result in gains in achievement. 

This study employed random assignment methodology to examine the impact of Guilford’s success 
course. The key findings presented in this report are: 

• Guilford’s implementation of its student success course stayed true to the On Course philoso- 
phy, with a strong emphasis on improving students’ psychosocial sk il ls and habits. 

• Challenges emerged during the study in maintaining instructors’ enthusiasm for teaching the 
course. 

• The course had a positive impact on students’ self-management, interdependence, self- 
awareness, interest in lifelong learning, emotional intelligence, and engagement in college 
among students with low levels of these attributes. 

• But the gains in efficacy did not lead to meaningful effects on students’ academic achievement 
during the program semester or in postprogram semesters. Despite the absence of an overall 
effect, the program did have positive effects on the first cohort of students enrolled in the study, 
with students demonstrating improved grades, retention in college, and credits earned. 

The results of this study reveal that improvements in students’ attitudes and behaviors may not 
necessarily translate easily into better academic outcomes, though the strength of program imple- 
mentation may play an important role in these effects. Additionally, the program’s limited effects 
suggest that community colleges should look to more comprehensive ways of improving develop- 
mental education students’ academic achievement, including reforms in developmental education 
instruction. 




Preface 



Community colleges have long been a crucial pathway into the middle class for low-income 
students who face barriers to education. However, these institutions also face a number of 
challenges in helping their students succeed, including historically low graduation rates and 
large numbers of students who enter college academically underprepared. The majority of 
community college students are often required to take a lengthy series of developmental, or 
remedial, education classes. Many never successfully progress through preparatory classes into 
a college-level curriculum. 

Given these challenges, many community colleges are experimenting with new meth- 
ods for improving developmental education students’ outcomes. Student success courses — 
also known as student development, study skills, or student orientation courses — are one 
popular method colleges have used to try to improve students’ understanding of college and 
their ability to navigate their way through school. This study, which analyzes the impacts of a 
student success course on developmental education students’ achievement at Guilford Technical 
Community College, provides one look at how these courses affect students’ achievement. 

The findings from this study reveal that Guilford’s success course was able to foster 
some changes in students’ attitudes and perspectives, such as their self-management, self- 
awareness, interest in lifelong learning, and engagement in college. Unfortunately, these 
improvements did not translate into improved academic outcomes for the overall group of 
students to whom the course was offered. 

Although the positive effects on the first cohort’s academic achievement are intriguing, 
the absence of an overall effect lends a more critical perspective on student success courses than 
have other recent quasi-experimental and experimental studies. While these courses have had a 
modest effect on student outcomes when paired with tutoring (as in Chaffey College’s Opening 
Doors program) or in learning communities (as at Kingsborough Community College), this 
study suggests that they may have more limited value when offered without such supports. 

While success courses may provide a positive benefit to students’ understanding of col- 
lege and its expectations, such courses may need to be more limited or integrated within larger 
stmctural changes in developmental education to improve students’ academic progress. 

Gordon L. Berlin 
President 
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Executive Summary 



Thousands of community college students across the country confront serious questions every 
day about their course-taking and career choices. They often are required to make key decisions 
relatively quickly, often with little introduction to how to make these choices most effectively. 
To further complicate matters, most community college students enter college academically 
underprepared and are required to take developmental, or remedial, English, math, and/or 
reading courses in order to advance to a college-level curriculum. ' 

Many colleges have looked to success courses,^ which seek to orient students to college 
life and assist them in making important decisions about college and their careers, as a way to 
help students address these challenges. In 2005, Guilford Technical Community College in 
Greensboro, North Carolina, joined many colleges across the nation in developing a success 
course for developmental education students as one of their strategies under the auspices of the 
Achieving the Dream: Community Colleges Count initiative. Achieving the Dream is a national 
organization designed to mentor colleges through an institutionwide, data-driven, student 
success-oriented improvement process.^ Along with developing students’ study habits, Guil- 
ford’s student success course placed an intensive focus on improving students’ psychosocial 
and soft skills, such as their awareness of their and others’ emotions, understanding of their own 
learning style, improved time management, and a recognition of their responsibility for their 
own learning. Guilford hoped that changing students’ personal habits and attitudes might help 
them take better control of their academic lives and improve their overall achievement. 

This report analyzes the impact of Guilford’s success course on developmental educa- 
tion students’ academic achievement as well as several social and psychological measures, 
including motivation, self-concept, and commitment to college. In sum, this study found that the 
success course had few effects on students’ academic achievement, though the evidence 
suggests that there were some positive, differential effects for students in the first group to enter 
the study. The success course was also found to have a positive impact on several psychosocial 
outcomes, including students’ self-management, interdependence, self-awareness, interest in 
lifelong learning, emotional intelligence, belief in self, and positive engagement in college. 



' Adelman (2004); Attewell, Lavin, Domina, and Levey (2006); Bailey, Joeng, and Cho (2010). 
^These courses are also termed study skills, student development, or new student orientation courses. 
^Achieving the Dream (2009). 
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Guilford’s Student Success Course 



Student success courses have become a popular strategy for increasing students’ achievement in 
community colleges throughout the country/ Guilford was particularly drawn to improving its 
developmental education students’ success rate after noting their poor academic performance in 
the college’s Achieving the Dream data analyses. In order to better address students’ socioemo- 
tional needs as well as their academic skills, Guilford chose to redesign an existing study skills 
course, shifting the course toward a greater focus on students’ responsibility for their own 
learning. Modeled on Skip Downing’s On Course philosophy,^ this newly revised course 
focused on helping students overcome their personal challenges through intensive reflection on 
their past history and future goals. Additionally, the course provided some opportunities for 
academic skill-building through class presentations, journal writing, quizzes, and an end-of- 
semester course project. The course was offered as a two-credit class to students in need of one 
or more developmental education course in reading, English, or math and was taught by faculty 
or staff members trained in the On Course philosophy and pedagogy. 

MDRC’s evaluation of Guilford’s success course began in spring 2008. It employs a 
random assignment evaluation design to examine the impact of the program on students’ 
achievement. A total of 91 1 students were enrolled in the study over the course of three semes- 
ters (spring 2008, fall 2008, and spring 2009). Using a lottery-like process, students were 
assigned to either a program group (458 students), which received the success course interven- 
tion, or to a control group (453 students), which received the college’s regular services (and 
were not enrolled in the success course). By comparing the outcomes of program and control 
group students over time, the study is able to gauge the impact of Guilford’s student success 
course on academic measures such as students’ persistence in college, grades, and course 
completion. Quantitative data on students’ backgrounds and academic achievement were also 
collected through a baseline information form and student transcript data. Qualitative data were 
gathered on the program and students’ experiences during two site visits to the college. During 
these visits, researchers interviewed administrators, faculty, staff, and participating students and 
conducted a student survey, which asked questions about the personal beliefs and habits the 
success course was expected to affect. 



Key Findings 

• Guilford’s implementation of the student success course stayed true to 
the On Course philosophy, with a strong emphasis on improving stu- 



"^Stovall (2000); Derby and Smith (2004); Zeidenberg, Jenkins, and Calcagno (2007); Scrivener, Sommo, 
and Collado (2009); Scrivener et al. (2008). 

^Downing (2008). 
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dents’ psychosocial s kil l s and habits. Instructors in the course were gener- 
ally well trained; all faculty and staff received at least a three-day introduc- 
tion to the On Course pedagogy and curriculum. Lessons tended to engage 
students in critical reflection about their own personal experiences and habits, 
with an effort to promote an awareness of their own role in their learning and 
future responsibilities. Students were encouraged to reflect upon their role in 
their learning through assignments such as weekly journal writing or by shar- 
ing their personal experiences with the class. Students also received some in- 
stmction and practice in study and academic skills through activities such as 
quizzes and a formal class paper and presentation. 

• Evidence suggests that the success course was more strongly implement- 
ed in the first semester of the study than in later semesters. During the 
first semester of the course, instructors had a great deal of enthusiasm for 
teaching the course and met monthly to discuss their teaching and best prac- 
tices. During the final two semesters, however, this enthusiasm was less ap- 
parent. Several new instmctors noted that they felt less prepared to teach the 
curriculum and deal with students’ socioemotional issues, while some sea- 
soned instmctors explained that they had seen few benefits from the course 
in their first semester of teaching it. Additionally, because monthly meetings 
were discontinued in the final two semesters, several new instmctors felt that 
they received less support in implementing the course. 

• Students’ participation in Guilford’s success course declined over the 
course of the program semester, resulting in only 61 percent of the pro- 
gram group students taking and completing the course. However, it is 
difficult to conclude how this may have affected students’ outcomes. 

Weeks or months often ensued between the time of random assignment and 
registration and the first day of classes, giving students ample time to reor- 
ganize their schedules and drop the class. When conducting exploratory 
analyses, the effect of student participation on academic outcomes was in- 
consistent over time. As a result, it is unclear whether a higher participation 
rate would have made the program more successful in improving academic 
outcomes. 

• A follow-up survey of program and control group students found that 
Guilford’s student success course had a positive impact on students’ self- 
management, interdependence, self-awareness, interest in lifelong learn- 
ing, emotional intelligence, and positive engagement in college among 
students with low levels of these attributes. This suggests that Guilford’s 
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success course was able to improve personal decision making, awareness of 
themselves and others, and commitment to their education among students 
who had lower levels of these abilities. 

• Guilford’s success course had no meaningful effects on students’ aca- 
demic achievement during the semester that the program operated or in 
the following semesters. There were no statistically significant differences 
between control and program group students in course registration or enroll- 
ment, credits attempted, course pass rates, course withdrawal rates, credits 
earned, successful completion of developmental education courses, or grade 
point averages, either during the semester of the success course or in the 
three following semesters. This lack of meaningful academic impacts sug- 
gests that the social and psychological impacts, though significant, may not 
have been strong enough to affect students’ achievement and that other pro- 
grams and services may be needed to improve their academic performance. 

The results of this study reveal that Guilford’s new success course, with its focus on so- 
cial and emotional skills, was able to foster some changes in students’ attitudes and perspec- 
tives. Unfortunately, however, these improvements did not translate into improved academic 
outcomes for the overall group of students to whom it was offered. While the mixed results 
from this study do not provide strong evidence that a success course alone improves student 
academic outcomes, these findings should be taken in concert with other studies that have 
shown more positive results for these courses, particularly when success courses are one 
component in more comprehensive programs. 

For instance, the findings in this study are not unlike those from a student success 
program at Chaffey College, which targeted probationary students. At Chaffey, an enhanced 
version of the student success program, which framed the program as mandatory and included 
visits to the college’s campus-based tutoring centers in reading, writing, and math, had 
positive impacts on students’ engagement and achievement.® Similarly, a number of positive 
impacts on students’ outcomes have been observed with learning communities that included a 
student success course. At Houston Community College, for example, a program that linked a 
success course and developmental math resulted in positive effects on students’ developmen- 
tal math pass rates.’ 

However, even when a program is well implemented, the impact of a one-semester in- 
tervention on students’ achievement may be modest and less likely to achieve the lofty goals of 



®Scrivener, Sommo, and Collado (2009). 
’Weissman et al. (201 1). 
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improved grades, credits earned, and graduation across the board. In each of the studies noted 
above, impacts on students’ academic achievement tended to be modest and generally sustained 
only during the semester in which the program operated or one semester after. These short-term 
effects suggest that other issues may be creating more substantial barriers to students’ academic 
progress than those a success course can address. For instance, while success courses may help 
improve students’ soft skills and give them a better understanding of how to manage college 
life, students may still be overwhelmed by larger challenges in their lives, such as an inability to 
afford their school tuition or the struggle to balance work, school, and family responsibilities. 
Similarly, success courses do not address some of the stmctural and academic barriers students 
may face in college, such as the need to pass a long sequence of developmental education 
courses or continued failure in a particular academic course. 

Success courses can play an important role in helping students to learn about the variety 
of programs and services that may benefit them during their academic career. However, these 
courses also present an opportunity cost for students, as they often enroll in a success course at 
the expense of taking an additional academic course. As this study found, students acquired 
fewer academic credits during the semester that they took the two-credit success course, in part 
because they were unable to fit in an additional academic course. An additional concern is that 
credits for most success courses are not transferrable to four-year institutions. 

With these concerns in mind, colleges may wish to consider ways in which success 
courses can be incorporated into larger, more systemic approaches to improving developmental 
education students’ academic experiences. Given the relatively short-term effects of these 
courses and the opportunity costs they present to students, colleges may need to look toward 
less ambitious interventions that support students’ psychosocial well-being but do not require 
them to choose between academic courses and their introduction to college life. In order to 
effect greater changes in students’ achievement, colleges might consider more comprehensive 
approaches to improving students’ academic performance, such as reforms in developmental 
education instmction or the stmcturing of developmental education course sequences. A 
different combination of these efforts, which address students’ social challenges while also 
focusing more concretely on their academic needs, may prove even more promising. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social and education policy research organization dedicated 
to learning what works to improve the well-being of low-income people. Through its research 
and the active communication of its findings, MDRC seeks to enhance the effectiveness of so- 
cial and education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy are- 
as and target populations. Once known primarily for evaluations of state welfare-to-work pro- 
grams, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Children’s Development 

• Improving Public Education 

• Raising Academic Achievement and Persistence in College 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




